Repository navigation
Conversation
7a21a97 to
313efb9
Compare
d8cbd45 to
118dba1
Compare
e0b46fd to
9ad6f02
Compare
|
Heads-up: at the current head ( The three |
d4574c1 to
240ed49
Compare
13ec9b7 to
2799511
Compare
|
TODO from the #1533 review: integration cases to add here. Should-fix 4: env re-sealing, which is not verified at runtime (#1539 removed the "locally verify" step from the protocol).
For each: let the guest push the replacement, wait for the VMM to drop the old file, restart the CVM, and check the decrypted env matches the original exactly. Should-fix 5: self-check freshness and relaxation switches (code unchanged; semantics documented in the KMS README by #1539).
|
1b95db6 to
60f0729
Compare
06ba15a to
6332786
Compare
0ff5221 to
3d4766b
Compare
Add the core-component acceptance suite under test-suites/: the dstack-test runner and dashboard, fixture providers, component harnesses, and acceptance cases for guest OS, VMM, KMS, gateway, verifier, and cross-component integration, together with suite documentation under docs/testing/. Signed-off-by: Kevin Wang <wy721@qq.com>
3d4766b to
e99ef93
Compare
Summary
This PR adds the core-component acceptance suite and organizes the complete suite directly under
test-suites/so specifications, automation, fixtures, inventories, manifests, and retained audit evidence have one clear home.dstack-testrunner, dashboard, registry validation, fixture providers, and component harnesses.case.mdspecifications asrun.py,run.sh, orrun.cjs.shared/automation/.metadata.jsonas the authoritative case-to-entrypoint binding.shared/fixtures/, inventories undercatalog/, environment manifests undermanifests/, and retained historical evidence underaudit/.Layout
Separation invariant
The PR targets
next. Its changed paths are limited to:REUSE.tomldocs/testing/**test-suites/**There are no product/runtime or source-local test changes under
dstack/**oros/**.Verification
prek run --all-files: passedreuse lint: passedgit diff --check origin/next...HEAD: passedRebase and tdxlab validation (2026-09-25)
nextat0fb3b24bbd, reviewing the 131 PRs merged sincee2cf39ae01. The per-PR mapping is intest-suites/audit/core-components-next-rebase-audit-2026-09-24.md.Admin.ImportCertreplacing the file-configured proxy certificate (refactor(gateway)!: replace the file-configured proxy certificate with Admin.ImportCert #1239), ZT-domain normalization (fix(gateway): normalize the domain in the remaining ZT-Domain admin RPCs #1356), image download retries (fix(verifier): retry transient image download failures #1388), collateral host allowlist (fix(pki-fetch): fetch certificate-named collateral only from allowlisted hosts #1404), setup-header layout (fix(os): normalize the kernel setup header to QEMU's layout, not zeros #1229), hotplug-off for GPU topologies (fix(vmm): turn PCI hotplug off by default and require it off for GPU passthrough #1387), and others.tc-kms-onboard-005,tc-gw-admin-037), andtc-vmm-vmm-021,tc-vmm-vmm-022andtc-vmm-compute-ne-006are retired with the registry pull. The suite now holds 373 cases, 372 of them scripted.Physical TDX (tdxlab, 4 workers, 372 scripted cases)
next(+ #1409 so images build)next635e7a2200+ #1414 + #1415The BLOCKED cases need an NVIDIA CC GPU on the TDX host or a Yocto image. The guest-side GPU cases were run on a GCP H100 CVM (a3-highgpu-1g) booting the candidate image and passed, apart from the reboot row that led to #1410.
Product fixes found while validating, sent separately
dstack-mrCLI--hotplug-offdefault does not match the VMM default (fix(vmm): turn PCI hotplug off by default and require it off for GPU passthrough #1387).systemd-tpm2-setupleft guests degraded.2775 tss:tssat runtime.StartVmissued while QEMU was still exiting afterStopVmwas silently skipped.All of these are merged into
next; #1409 and #1413 were closed in favour of #1415.tc-kms-onboard-005, which depends on #1406, is now promoted.Local gates
Registry verification (373 total, 372 scripted, 372 promoted, 0 findings), 40 runner unit tests,
reuse lint,prek run --all-files, andgit diff --check origin/next...HEAD.Rebased onto #1534 (2026-11-08)
The branch was rebased from its actual fork point
0fb3b24bbd94c18d4af2900b41cd82bcd1c0c284onto #1534 (feat/kms-epochs-three-commits), rather than treating the currentnexttip as the source base.Added
tc-kms-rotation-013, a focused root/app key rotation lifecycle matrix covering keyring import, next-epoch generation, activation and serving, handover chain-hash checks, epoch-scoped authorization, restart recovery, disk epoch migration, and SealedEnv epoch binding. The matrix contains 14 focused Rust rows and rejects stale, forked, incomplete, or cross-epoch state.First tdxlab rotation round: 14/14 rows passed on the tdxlab host on November 8, 2026, using Rust 1.92.0 and local Cargo targets. This round exercised the focused unit matrix; it did not boot a TDX VM or perform a live KMS deployment. Registry verification reports 373 scripted, 373 promoted, 374 total cases, with no findings. Prek passed for the added and modified files.
Forward-interval coverage audit (2026-11-08)
The earlier rotation-only addition was incomplete. This revision adds:
tc-kms-rotation-013: 14 Rust rows for keyring import/generation, activation and serving, handover validation, historical epochs, restart recovery, authorization selection, disk epoch migration, and SealedEnv epoch binding.tc-kms-rotation-014: complete Foundry contract matrix for root/app epoch registration, chain hashes, activation boundaries, pending removal, evidence, owner checks, stale selectors, revocation, and upgrade compatibility.tc-kms-rotation-015: auth-simple, auth-mock, auth-eth-bun, and auth-eth backend suites for operation- and epoch-scoped authorization.The coverage audit is recorded in
test-suites/audit/pr1534-forward-coverage-2026-11-08.md. The first tdxlab Rust round remains 14/14 PASS. tdxlab currently has neither Foundry nor the backend JavaScript dependencies, so contract and backend cases are intentionally not promoted or claimed as executed there.Additional non-rotation coverage (2026-11-08)
Added
tc-int-forward-016for focused regressions from the forward interval outside key rotation: NUMA and extra-root-device measurement, VM annotations, passt/QMP behavior, guest container-name validation, RA-TLS key possession, TPM response bounds, and dstack-types serialization. The case deliberately fails on a zero-test result and records the live-vs-unit boundary.The three newly added matrix cases (
tc-kms-rotation-014,tc-kms-rotation-015, andtc-int-forward-016) remain unpromoted until their required Foundry, JavaScript, or full runtime prerequisites are available. The registry therefore reports them explicitly as pending rather than claiming execution.tdxlab execution completed (2026-11-08)
After installing Foundry 1.9.0, Node.js 22.23.4, and the locked JavaScript dependencies on the tdxlab host:
tc-kms-rotation-013: 14/14 Rust rotation and migration rows passed.tc-kms-rotation-014: 82/82 Foundry contract and upgrade rows passed (DstackKms.t.sol42,DstackApp.t.sol28,UpgradesWithPlugin.t.sol12). Foundry artifacts were cleaned between suites.tc-kms-rotation-015: auth-simple 33, auth-mock 15, auth-eth-bun 50, and auth-eth 22 tests passed under Node.js 22.23.4.tc-int-forward-016: 11/11 focused non-rotation rows passed: NUMA layouts, extra root devices, VM annotations, passt/QMP, guest container-name validation, RA-TLS key possession, TPM response bounds, and dstack-types serialization.The tdxlab host does not have a GPU or a live deployed KMS/Gateway chain in this round, so these results cover the deterministic contract, backend, Rust, and host-side matrices; live TDX VM lifecycle and deployed-chain rotation remain outside this execution round. The registry now reports 376 scripted, 376 promoted, 377 total cases, and zero findings.
Physical-TDX rotation migration result (2026-11-08)
The corrected physical-TDX harness passed
tc-int-compatibil-008on the latest runkms-e2e-20261108-fix18:The run also passed the existing physical mixed KMS upgrade path once (
tc-int-mixed-002, runkms-e2e-20261108-fix11). Earlier failures were retained as diagnostic runs while correcting test version pairing, signer pin isolation/normalization, Python SealedEnv dependencies, and the 0.6.0 image preparation.Final 0.6.0 rotation migration verification (2026-11-08)
The latest physical-TDX run
kms-e2e-20261108-fix17passedtc-int-compatibil-008:The hardware preparation script now checks
cryptography,eth_keys,eth_utils, andpyhpkebefore starting a build, so missing SealedEnv dependencies fail during preparation instead of during VM deployment.Rebased onto next at #1587 (2026-11-15)
Rebased from
9b87d9f9a6ontonextataa8805a917(#1587). The per-PR mapping is intest-suites/audit/core-components-next-rebase-audit-2026-11-15.md.[listeners.*](refactor(kms)!: group listener settings under [listeners.*] #1585, refactor(gateway)!: group listener settings under [listeners.*] #1587), the PROXY header tests inrocket-proxy-listener(feat(kms,gateway): accept the PROXY protocol and log the client address #1582), the TDX lite tests indstack-mr(feat: expose TDX lite OS check as a self-contained wasm package #1555), the registered root CA and root epoch in the upgrade-policy webhook (feat: KMS-enforced launch tickets #1577), and the KMS upgrade matrix for a candidate that onboards only from 0.5.11 or newer (feat(kms): add root/app key epochs, SealedEnv and epoch-scoped authorization #1533).tc-gos-platform-010(the candidate row gets its keys from a candidate KMS in a CVM), the verifier's HTTP 500 for an unprocessable request (fix(verifier): coalesce AMD collateral fetches and report dependency outages #1484), and Foundry builds into case-owned directories.tc-kms-kms-007(KMS listeners, public listener, PROXY protocol, audit lines),tc-gw-internal-009(gateway listener config, PROXY protocol, admin audit and access lines),tc-int-forward-017(24 product test rows for launch tickets,dstack-mr/verifier-wasm, SEV-SNP and listener config).tc-gos-build-001andtc-kms-rotation-014cover Merge release/0.6.3 (and 0.6.2) into next #1566 and feat(kms-auth)!: pass the TEE platform to policies and allow restricting it #1567.Product fixes found while validating: #1590 (a simulated TDX guest got a no-TEE vm_config, so the KMS refused its keys) and #1565 (
get-keysverified the KMS against production collateral only).tdxlab (4 workers, 379 scripted cases, this branch + #1565 + #1590)
The BLOCKED cases need an NVIDIA CC GPU. The two reruns are
tc-int-failure-se-001(observer listener timeout) andtc-gos-observabil-002(host API port taken between reservation and bind).